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ABSTRACT 


A  group  of  51  adults  was  studied  to  determine  if  multiple  patterns  improved  the 
overall  recognition  rates  of  a  selected  retinal  scan  device.  The  experimental  mode  was 
a  laboratory  simulation  of  a  locking  door  mechanism  controlled  via  The 
EyeDentification  System  7.5.  The  study  examined  single,  double,  or  triple  patterns  of 
individual  participants  over  the  course  of  eight  weeks  to  see  if  any  improvement  in 
successful  recognition  rates  could  be  obtained.  After  four  weeks,  the  system  achieved 
100%  successful  recognition  rates  over  all  three  groups  regardless  of  the  number  of 
templates  in  memory.  No  false  acceptances  were  recorded  for  the  experiment.  The 
conclusion  drawn  from  this  study  was  that  the  use  of  multiple  patterns  versus  the  use 
of  a  single  eye  template  reference  pattern,  with  The  EyeDentification  System  7.5,  did 
not  significantly  increase  the  overall  success  rate  of  recognition  for  a  given  population. 
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I.  INTRODUCTION 


The  current  surge  in  technologies  in  the  field  of  biometrics  has  increased  public 
awareness  of  the  realm  of  possibilities  and  has  manufacturers  producing  a  range  of  new 
products.  Biometrics  is  defined  as  the  use  of  unique  personal  characteristic(s)  to 
identify  an  individual.  Via  biometric  devices,  a  person  is  identified  "by  virtue  of  what 
he  is  rather  than  what  he  has  (a  key,  a  number,  or  an  ID  card)."  [Ref.  1:  p.  78] 

The  theory  behind  the  new  biometric  technologies  is  that  it  is  much  harder  to 
falsify  retinal  patterns,  fingerprints,  palm  prints,  or  voice  patterns  than  to  duplicate 
card  keys,  personal  identification  numbers  (PIN),  or  photo  access  badges  that  are 
separate  and  easily  misplaced.  Any  additional,  non-biometric  security  measure  often 
requires  just  another  item  to  keep  up  with  in  a  daily  operational  mode.  Often  cipher 
locks  are  successfully  navigated  after  taking  the  combination  out  of  a  wallet  or  pocket. 
Use  of  a  biometric  device  requires  that  the  individual  is  physically  present  and  is 
familiar  with  operational  procedures. 

Greater  effectiveness  of  any  security  measure  is  a  necessity  in  today's  world  of 
sensitive  materials  and  protected  communications.  A  variety  of  biometric  technologies 
are  being  offered  on  the  commercial  security  market  today  and  more  investigation  into 
the  protocols  and  processes  of  fully  implementing  a  system  of  any  type  becomes  a 
necessity.  [Ref.  2:  p.  6] 

Beta  testing,  confirmation  that  the  designed  software  and  hardware  produce  the 
desired  (and  future  advertised)  effect,  is  usually  completed  before  the  product  is  offered 
on  the  market.  Testing  for  protocols  and  procedures  encompasses  the  actual  use 
within  the  specific  operational  setting  and  all  interim  refinements.  The  resulting 
guidelines/procedures  conform  to  all  managerial  expectations  and  thresholds. 
Depending  on  site  specific  requirements  and  rising/lowering  expectations,  the  "final" 
procedures  can  be  a  drawn  out  and  many  looped  iterative  process. 

This  experiment  examines  the  managerial  implications  of  using  a  single  personal 
identification  device  in  a  simulated  controlled  work  environment.  The  biometric 
parameter  chosen  for  the  study  was  the  retinal  blood  vessel  pattern,  [Ref.  3:  p.  238]  due 
to  the  retina's  unique  and  relatively  unalterable  pattern. 
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TABLE  I 

COMPARISON  OF  BIOMETRIC  SECURITY  METHODS 


ERROR 

UNIT 

biometric 

measure 

METHOD 

RATES 

COST 

FINGER¬ 

PRINT 

optical 

digitization 

low 

3.5-7K 

HAND 

PRINT 

photo 

sensitive 

medium 

7-8K 

RETINAL 

SCAN 

infra-red 

heat 

detection 

very  low 

10K 

PALM 

PRINT 

optical 

digitization 

low 

12K 

VOICE 

PRINT 

contour 

voiceprint 

low 

5-20K 

SIGNATURE 

DYNAMICS 

pressure/ 

acceleration 

low 

2-5K 

*  table  adapted  from  Government  Computer  News,  July  19,1985 


One  manufacturer,  EyeDentify  Inc.  of  Beaverton,  Oregon,  currently  markets  a 
retinal  scan  device  for  use  in  physical  security  applications.  The  EyeDentification 
System  7.5  model  was  chosen  for  use  on  a  sample  of  young  adults  at  the  Naval 
Postgraduate  School. 

Other  physical  features,  as  seen  in  Table  I,  are  unique  and  can  be  used  as  a 
means  of  control.  None  of  the  other  measures  are  as  stable  and  unchanging  as  the 
retinal  vascular  pattern.  The  eye  shares  the  same  stable  environment  as  the  brain. 
Only  a  few  diseases  or  injuries  are  serious  enough  to  alter  the  blood  vessel  pattern  of 
the  retina.  These  rare  occurances  would  probably  place  the  individual  in  a 
non-working  status  and  not  change  the  overall  effectiveness  of  the  biometric  device. 
An  indirect  benefit  of  use  of  the  retinal  scan  could  be  as  a  rudimentary  diagnostic  tool. 
If  problems  with  identification  of  an  individual  by  the  retinal  scan  surfaced  after  a 
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Figure  1.1  Retinal  Scan  Diagram. 


period  of  successful  use,  one  possible  conclusion  could  be  the  onset  of  disease  serious 
enough  to  alter  blood  vessel  patterns. 

Biometric  devices  and  their  usefulness  to  security  applications  depends  on  the 
capability  of  comparing  an  image  of  a  physical  characteristic  with  a  stored  record  of 
that  image. 


FycDentify's  retinal  scanning  method  starts  with  a  450-degrce  circular  scan  of  the 
portion  of  the  retina  centered  on  the  fovea,  or  the  area  of  sharpest  vision.  A 
low-intensity  infrared  light  is  used  by  a  camera  (called  the  I  CAM)  to  make  the 
retinal  scan.  As  it  scans  the  area,  the  camera  takes  320  readings,  measuring  the 
variations  in  reflections  that  indicate  the  pattern  of  .blood  vessels.  A 
software-implemented  phase  corrector  compensates  for  rotations  of  the  head  or 
eye  by  the  user.  The  scan  by  the  ICAM  produces  a  waveform  that  is  converted 
into  digital  impulses  and  sent  to  a  microprocessor.  [Ref.  4:  p.  1-5] 


Figure  1.1  illustrates  the  specific  area  of  the  retina  that  the  EycDentification 
System  7.5  ICAM  measures  to  produce  the  eventual  stored  eye  template.  The  stable 
environment  of  the  eye  should  provide  nearly  identical  patterns,  if  the  ICAM  is  focused 
on  the  same  area  for  each  trial.  Multiple  eye  templates,  when  taken  under  the  same 
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enrolling  protocols  or  procedures  by  the  same  system  operator,  merely  reflect  minor 
head  alignment  variations  that  should  not  affect  overall  recognition  capability. 

This  experiment  was  based  on  the  premise  that  one  set  of  managerial  guidelines 
is  used  to  find  the  best  device  (biometric  or  not)  to  start  or  add  to  current  security.  A 
completely  different  set  of  managerial  guidelines,  procedures  or  protocols  is  required  to 
know  how  to  use  any  device  as  efficiently  and  effectively  as  possible  on  the  specific  site. 
The  decision  points  and  pathways  to  developing  in-house  protocols  by  a  management 
system  operator  are  based  on  observed  performance  of  individuals  using  the  system. 

The  author's  use  of  protocol  or  protocol  materials  refers  to  the  preliminary  draft 
procedures  of  mandated  in-house  usage  at  the  specific  site  of  installation.  Using  this 
definition  protocols  will  vary  from  installation  to  installation  depending  on  use  as 
specified  by  individual  managers.  Procedures  and  protocols,  until  finally  in  place,  are 
the  result  of  an  iterative  process  between  the  users  (of  the  system)  and  the  overseeing 
manager.  As  the  organization  or  security  requirements  change,  any  established 
protocol(s)  will  have  to  be  evaluated  and  reworked  as  necessary. 

This  experiment  observed  51  adults  and  compared  results  with  other  commercial 
installations  currently  using  The  EyeDentification  System  7.5.  Comparison  of  the 
experimental  results  with  the  site  responses  are  discussed  in  detail  in  the  conclusions. 
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II.  DESCRIPTION  OF  PROBLEM 


Several*  previous  studies  had  used  retinal  scans  to  study  small  groups  under 
highly  controlled  conditions.  [Ref.  5,6:  pp.  42,  57]  These  studies  established  the 
feasibility  of  using  the  retinal  scan  for  physical  security.  This  study  investigated  the 
effective  use  of  various  managerial  protocols  in  conjunction  with  the  retinal  scan  and 
sought  ways  to  increase  the  probability  of  obtaining  a  successfully  recognized  scan. 

Effective  "on-the  job"  retinal  scan  usage  should  entail  a  manageable  system  that 
allows  self  service  of  identification.  When  used  in  conjunction  with  a  locking  door 
mechanism,  ideal  usage  would  produce  no  false  rejects  of  authorized  personnel  (Type  I 
error)  and  not  admit  any  unauthorized  person  (Type  II  error).  Type  I  error  refers  to 
the  mathematical  probability  that  a  person  enrolled  in  the  system  and  an  authorized 
user,  will  be  rejected  or  not  identified  as  an  authorized  user.  Type  II  error  refers  to 
authorized  users,  unauthorized  users  or  imposters  that  are  incorrectly  identified  as 
another  authorized  user. 

A  manager's  nightmare  is  a  marketed  complete  physical  security  mechanism  that 
would  allow  access  to  unauthorized  personnel  (Type  II  error).  Additional  managerial 
concerns  are  the  authorized  workers  who  are  held  in  an  isolation  booth  or  are  unable 
to  get  through  a  locked  door  and  become  frustrated  (Type  I  error).  Effective  managers 
would  choose  to  have  a  system  that  produces  the  least  possible  combination  of  Type  I 
and  II  error.  Unauthorized  entry  problems  (Type  II  error)  is  a  far  more  critical 
security  problem  for  managers  and  is  treated  with  different  procedures  than  employee 
frustrating  Type  I  error. 

This  experiment  sought  to  explore  individual  usage  patterns  and  the  effect  of 
multiple  eye  signature  patterns  in  memory  when  used  daily  to  simulate  the  controlled 
entry  to  work  spaces.  Although  both  recognition  and  verification  modes  were  available 
on  the  System  7.5  ,  only  the  recognition  mode  was  used  in  the  study. 

Recognition  mode  on  the  System  7.5  checks  the  'new'  retinal  scan  against  all 
other  patterns  contained  in  memory  for  a  positive  match.  In  other  words,  the  just 
scanned  retina  must  match  another  pattern  in  memory  for  the  individual  TO  BE 
recognized.  The  verification  mode  operates  by  the  machine  comparing  the  Personal 
Identification  Number  (PIN)  entered  with  the  stored  retinal  pattern  in  memory  for  that 
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PIN.  Operating  with  the  verification  mode  does  not  remove  the  hazard  of  an 
individual  writing  down  the  PIN  on  a  slip  of  paper  to  carry  in  a  wallet  because  the 
biometric  parameter  is  not  the  sole  source  of  identification. 

Recognition  mode  was  determined  to  most  closely  simulate  working  conditions 
with  positive  identification  of  the  individual  scanned  as  compared  to  verification  of  a 
personal  identification  number  (PIN)  in  the  verification  mode.  Both  modes  on  The 
EyeDentification  System  7.5  utilize  the  40-byte  eye  signature  template.  Baseline  system 
parameter  is  prescribed  by  the  manufacturer  for  recognition,  as  a  correlation  score  of 
75%  between  reference  templates,  or  individual  eye  signature(s)  in  memory,  and  the 
newly  acquired  fresh  scan.  The  lowest  correlation  scores  recommended  by  the 
manufacturer  are  those  that  are  pre-set  at  the  factory  (75%  for  recognition,  70%  for 
verification).  [Ref.  4:  p.  2-12]  Individual  sites  have  the  option  of  going  in  through  the 
change  menu  and  raising  or  lowering  the  threshold  based  on  degree  of  sensitivity 
required  by  that  particular  facility. 

The  recognition  mode  compares  the  freshly  acquired  scan,  taken  at  the  press  of 
the  scan  button,  with  the  stored  reference  templates  of  all  individuals  currently 
contained  in  memory.  A  successful  match  against  the  databank  produces  the 
PERSONAL  IDENTIFIER  from  that  eye  signature  in  the  window  of  The 
EyeDentification  System  7.5.  (See  Figure  2.1)  The  PERSONAL  IDENTIFIER  is 
normally  the  full  name  of  the  individual  or  the  last  name  and  first  initial  of  the  person 
enrolled.  Any  department  name  or  other  identifying  label  up  to  eight  (8)  characters 
could  be  entered  into  memory  as  deemed  necessary  by  the  system  operator  or 
overseeing  manager. 

Coupled  with  a  locking  door  mechanism,  the  successful  match  would  also  unlock 
the  interior  access  door  of  the  isolation  booth.  An  unsuccessful  match  produces  the 
message  REPEAT  which  indicates  that  no  match  was  found  and  tells  the  individual  to 
repeat  the  scanning  process.  With  an  unsuccessful  match  the  door  remains  locked. 
Three  consecutive  unsuccessful  attempts  are  allowed  before  the  message  SEE 
SECURITY  is  displayed  in  the  window.  At  this  point  a  "panic"  button,  or  other  device 
to  alert  security/management,  is  necessary  for  the  individual  to  get  through  the 
controlled  access  door.  By  law,  all  access  devices  have  to  allow  the  individual  to  easily 
retrace  attempted  entry  path  if  access  is  blocked  due  to  failed  identification  or 
recognized  unauthorized  user. 
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Figure  2.1  The  EyeDentification  System  7.5. 

An  ideal  management  goal  would  be  to  flawlessly  operate  the  scan  with  zero 
percent  type  1  and  II  error  for  maximum  security  and  minimum  employee  frustration. 
Operating  with  an  advertized  Type  1  error  rate  of  1%  [Ref.  1:  p.  78],  The  System  7.5 
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has  an  established  success  rate  of  99%  for  recognizing  enrolled  retinas  correctly.  With 
additional  protocols,  a  realistic  managerial  expectation  would  be  to  operate  the  retinal 
scan  with  a  successful  recognize  rate  of  above  99%.  The  area  of  this  investigation  was 
to  examine  whether  Type  I  and  Type  11  errors  were  affected  by  the  use  of  one  or  more 
eye  signature  reference  templates  in  memory. 
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III.  DESIGN  AND  METHODOLOGY 


A  request  for  volunteers  for  a  thesis  experiment  was  circulated  in  two 
subpopulations  of  the  Naval  Postgraduate  School,  Monterey,  California.  Operations 
Research  (OR)  class  3404  and  section  PL-53  of  the  Computer  Curriculum  provided  a 
majority  of  the  human  subjects  for  the  experiment.  Due  to  the  nature  of  the 
population  at  the  Naval  Postgraduate  School  (NPS)  and  the  method  of  invitation  to 
participate,  this  study  cannot  be  considered  free  of  bias  of  the  school  population  as  a 
whole,  but  with  the  uniformity  of  the  students,  it  is  likely  this  is  a  representative 
sample. 

Excellent  general  health  and  eyesight  that  is  correctable  to  20-20  are  basic 
underlying  attributes  of  the  Naval  population  at  large  that  provided  the  majority  of  the 
baseline  personnel  pool  for  the  overall  testing  pool  of  subjects.  Three  (3)  foreign 
officers,  two  (2)  civilians  (one  staff  and  one  NPS  student),  and  46  Department  of 
Defense  (DOD)  NPS  students  comprised  the  group  that  volunteered  for  the 
experiment. 

Age  range  studied  was  25  to  40  years  in  this  experiment.  Thirty-nine  (39)  male 
and  twelve  (12)  female  participants  were  studied  over  a  nine  (9)  week  period.  (See 
Table  II) 

This  original  group  of  51  subjects  was  broken  down  into  three  (3)  subgroups  for 
the  purposes  of  experimentation.  Group  III  (Appendix  A)  was  randomly  selected  from 
the  original  starting  group  and  was  re-enrolled  on  two  (2)  separate  occasions  during 
the  run  of  the  experiment.  The  first  re-enrollment  for  Group  III  took  place  at  week 
four  (4)  or  about  the  one  third  mark  of  the  time  period.  A  second  re-enrollment  was 
taken  during  week  six  (6)  or  approximately  a  two  thirds  mark.  Seventeen  individuals 
were  selected  for  Group  III  participation. 

Group  II  (Appendix  B)  was  also  randomly  selected  from  the  original  group  but 
was  only  re-enrolled  once  during  the  experiment,  at  about  the  halfway  point  or  during 
week  five  (5).  An  additional  seventeen  individuals  were  selected  and  tagged  for  Group 
II  from  the  remaining  thirty  four  people  left  of  the  original  starting  group. 
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TABLE  II 

DIFFERENT  PHASES  OF  THE  EXPERIMENT 


GROUP  III 


first 

re-enroll 


second 

re-enroll 


GROUP  II 


Irst 

re-enroll 


GROUP  I 


Phase  ONE  Phase  TWO  Phase  THREE  Phase  FOUR 


Group  I  (Appendix  C),  those  seventeen  individuals  not  randomly  placed  in  one 
of  the  other  groups  acted  as  a  control  group  and  provided  baseline  data  for 
comparison  purposes.  Individuals  in  Group  I  had  a  single  enrollment  at  the  start  of 
the  experiment  and  were  never  re-enrolled. 

Experimenter  desired  baseline  success  level  for  the  retinal  scan  experiment,  as 
outlined  in  the  experimental  description  section,  is  a  99%  recognition  rate.  Each 
participant  would  daily  attempt  to  scan  and  log  all  occurances  for  the  eight  week 
period.  Upon  completion  of  the  experiment,  each  group  would  be  statistically  analyzed 
for  usage  trends  and  behavioral  patterns. 

All  subjects  were  given  an  introductory  brief  description  of  what  the  retinal  scan 
was,  how  operation  was  performed  and  why  it  was  most  important  to  get  the  best 
patterns  at  the  enrollment  process  because  the  resulting  template  would  affect  the 
success  rate  of  future  use.  Enrollment  was  conducted  in  the  Man-Machine  Interface 
(MM I)  Laboratory  on  The  EyeDentification  System  7.5  (model  #  100519)  with  head 
alignment  bar  and  luminous  dot  target.  Over  a  two  (2)  day  period  all  51  individuals 
were  enrolled  in  a  sequential  order  determined  by  their  respective  personal  schedules. 
Although  sequentially  arranged  in  the  bubble  memory  the  participants  were  not  aware 
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they  were  assigned  a  Personal  Identification  Number  (PIN)  or  that  they  were  arranged 
sequentially. 

Participants  were  instructed  to  take  a  seat  and  look  into  the  eye  lens  while 
arranging  their  hands  on  either  side  of  the  machine  with  their  right  thumb  lightly  over 
the  scan  button.  After  finding  the  light  source(s)  with  their  right  eye,  they  were 
instructed  to  align  all  observed  sources  so  that  the  intense  pin  point  of  light  was  in  the 
exact  center  of  any  blob  or  circle(s)  of  light.  Five  individual  retinal  patterns  with  a 
correlation  of  90%  or  higher  were  used  to  produce  a  single  averaged  reference  eye 
signature  pattern  for  each  individual,  which  was  stored  in  memory,  and  served  as  the 
master  pattern  against  which  all  fresh  scans  were  compared.  Any  subsequent 
re-enrollment  followed  the  same  procedure  of  five  patterns  with  a  correlation  of  90% 
or  higher. 


Enrollment  is  simply  the  process  of  creating  and  storing  an  individual's  eye 
signature  template.  Each  eye  scan  is  built  from  a  minimum  of  three  to  five 
retinal  scans.  Each  scan  is  stored  by  the  system,  based  on  how  it  compares  with 
the  previous  scans.  The  better  the  comparison,  the  higher  the  score.  The  eye 
signature  template  created  during  individual  enrollment  is  the  key  to  consistent, 
positive  identification  of  each  authorized  individual.  This  template  allows  the 
system  to  recognize  an  enrollee,  while  rejecting  any  imposter  who  may  have  a 
similar,  but  not  identical,  eye  signature. .  usually,  an  organization  appoints  one 
person  to  act  as  the  system  operator.  High  quafity  eye  signatures  are  the  key  to 
successful  operation  of  the  EyeDentification  System  7.5.  [Ref.  4:  p.  2-7] 


The  software  component  of  the  7.5  drives  the  process  of  enrollment.  An  eye 
signature  template  is  built  from  several  ICAM  (eye  camera)  scans  for  each  individual 
as  they  are  enrolled  in  the  system.  Reflected  light  from  the  infrared  ICAM  is  collected 
and  converted  into  an  analog  signal.  The  analog  waveform  is  digitized  and  stored  in 
memory.  The  System  uses  forty  (40)  bytes  (320  bits)  of  the  enrolled  template  in 
memory  to  successfully  reference  the  desired  template  per  individual  or  eye  in  PIN 
verification  mode,  or  72  bytes  (576  bits)  per  individual  or  eye  in  recognition  mode. 
These  differing  bit  templates  are  system  requirements  and  are  a  function  of  the 
software  processing  to  either  recognize  or  verify  against  the  original  stored  eye 
signature  templates.  Once  this  eye  signature  is  built,  it  is  linked  with  other  individual 
identification  information  such  as  name,  department,  or  other  data,  and  stored  in  the 
IBANK  (enrollee  databank)  memory  along  with  other  eye  signatures.  On  the  initial 
enrollment,  usually  more  than  five  patterns  were  attempted  but  only  those  patterns 
meeting  the  criteria  of  a  correlation  of  90%  or  better  for  each  retinal  eye  signature 
template  attempted  were  used  in  the  enrolling  portion  of  the  experiment. 


17 


Few  difficulties  occured  at  initial  enrollment.  Most  questions  and  concerns  were 
attributed  to  the  learning  curve  of  a  new  piece  of  machinery  or  with  the  potential  eye 
health  problems  with  shining  a  light  in  the  eye.  Use  of  the  EyeDentification  System 
7.5  poses  no  danger  or  risk  to  the  individual  of  physical  injury.  The  source  of 
illumination 'is  a  gallium  aluminum  arsenide  infrared  light-emitting  diode.  [Ref.  4:  p. 
1-6]  The  levels  of  light  emitted  during  scanning  are  several  orders  of  magnitude  below 
the  safety  standards  established  by  the  American  Conference  of  Governmental 
Industrial  Hygienists.  Products  using  this  same  technology  include  refrigerator  door 
lights,  smoke  detectors,  television  remote  controls,  and  wireless  headsets. 

Noted  by  the  system  operator  was  the  ease  of  use  by  artillery  and 
bombardier-navigator  (BN)  officers  who  had  never  used  a  retinal  scan  but  had 
familiarity  with  sighting  equipment.  This  concentration  of  individuals  with  such 
experiences  would  not  be  possible  in  an  unbiased  population  but  will  not  unduely 
skew  any  overall  results  of  the  experiment. 

As  subjects  were  re-enrolled,  their  new  pattern  and  PIN  were  stored  in  bubble 
memory  without  deleting  any  previous  pattern(s).  Eighty  (80)  templates  were  already 
contained  in  bubble  memory  at  the  beginning  of  the  experiment  from  previous, 
unrelated  work.  Consequent  enrollment  of  the  entire  original  group  caused  the  enrollee 
databank  to  grow  to  131.  Re-enrolling  Group  III  during  phase  TWO  of  the 
experiment  brought  the  I  bank  total  to  144  by  adding  13  second  templates.  Only 
thirteen  templates  were  added  on  the  first  re-enroll  of  Group  III.  Four  of  the  selected 
group  did  not  re-enroll  despite  numerous  scheduled  re-enrollment  sessions.  After  a  two 
week  period  of  unsuccessful  total  group  re-enrollment,  Group  III  was  redefined  to  be 
the  thirteen  (13)  individuals  who  had  re-enrolled  and  were  willing  to  re-enroll  during 
week  six. 

Phase  THREE  added  17  re-enrollments  from  Group  II  to  bring  the  total  to  161 
eye  signatures  in  memory.  No  problems  were  encountered  with  obtaining 
re-enrollments  on  the  randomly  selected  seventeen  individuals  from  the  original  51. 
Group  II  remained  as  chosen  during  the  entire  experiment  without  any  modifications 
or  personnel  shifts. 

The  last  re-enrollment  of  Group  III,  with  their  13  templates,  in  Phase  FOUR 
brought  the  total  to  174  templates  in  I  bank  memory.  As  the  study  progressed  through 
the  four  phases,  the  number  of  templates  against  which  any  fresh  scan  was  compared 
did  not  remain  constant  during  the  eight  weeks.  An  additional  two  (2)  templates  were 
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added  to  the  Ibank  memory  by  re-enrolling  two  individuals  that  were  having  a  difficult 
time  getting  recognized  by  the  7.5  during  the  first  weeks  of  the  experiment.  At  the 
completion  of  the  Phase  FOUR,  the  176  total  templates  were  the  result  of  the 
scheduled  174  templates  from  the  re-enrollments  of  Groups  II  and  III  plus  the  two 
unscheduled  re-enrolls  to  assist  individuals  with  their  potential  access  problems. 

Individuals  were  logged  on  a  daily  log  sheet  (see  Table  III)  so  progress  could  be 
watched  and  determination  could  be  made  how  patterns  were  being  matched  in  the 
Ibank.  Listed  participants  only  had  to  record  their  attempts  for  daily  trials  beside  their 
name. 


TABLE  III 

FIRST  WEEK  LOG-IN  SHEET 


WEEK:  1 

first  trial 

second  trial 

name 

SCAN  ATTEMPT 
"the  message  or 
personal  identifier 
displayed  as  a 
result 'of  the  scan 
attempt" 

SCAN  ATTEMPT 

ADCOCKJ 

adcockj 

*R,  adcockj 

AGAPIOUNE 

**R,  R,  see  security 

R,  agapioune 

ALDERMANB 

aldermanb 

aldermanb 

BABBR 

babbr 

R,  R,  babbr 

*  R  represents  the  single  REPEAT  message  followed  by  a  success 
**  multiple  R's  and  SEE  SECURITY  represents  an  unsuccessful  attempt 


The  experiment  consisted  of  an  individual  entering  the  lab,  looking  through  the 
eye  lens,  finding  the  light  source(s)  and  aligning  the  head  correctly  before  pressing  the 
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scan  button.  The  EyeDentification  System  7.5  was  set  in  the  recognition  mode  for  the 
duration  of  the  study,  so  that  on  attempted  entry  an  individual's  fresh  scan  was 
compared  against  all  templates  in  memory. 

When  thd  EyeDentification  System  is  in  recognition  mode,  it  checks  all  templates 
in  memory  m  search  of  a  match.  Identification  time  depends  on  the  number  of 
eye  signature  templates  stored  in  memory.  You  can  tell  if  your  unit  is  in 
recognition  mode  by  looking  into  the  eye  lens.  If  the  dot  alignment  target  is 
present,  the  unit  is  in  recognition  mode.  To  gain  access  when  the  unit  is  in 
recognition  mode,  look  into  the  eye  lens  and  concentrate  on  the  dot  alignment 
target  visible  inside  the  ICAM.  When  the  smallest  brightest  dot  is  centered within 
the  larger  dots,  press  the  SCAN  button.  The  system  then  compares  the  resulting 
wave  form  witn  all  stored  eye  signatures.  The  person  is  recognized  or  not, 
depending  upon  whether  or  not  a  matching  signature  is  Tound.  Upon 
recognition,  the  system  automatically  activates  release  of  the  security  mechanism 
involved,  [kef.  4:  p.  2-30] 

Phase  ONE  of  the  experiment  was  to  complete  the  initial  enrollment  process  and 
start  everyone  on  a  five  day  per  week  schedule  of  two  (2)  attempted  recognitions  a  day. 
Daily  use  was  defined  to  be  at  least  two  (2)  attempts  sometime  during  the  day.  A 
total  of  ten  (10)  attempts  were  logged  for  a  complete  week  for  each  subject.  All  three 
groups  had  a  single  template  in  memory  and  were  following  the  exact  same  scan  and 
log-in  sheet  procedures. 

This  initial  phase  was  designed  to  get  all  participants  used  to  the  system  and  its 
successful  use  and  providing  baseline  data.  Three  weeks  of  the  Phase  ONE  operation 
would  do  away  with  any  learning  curve  effects  that  might  affect  later  phases.  During 
Phase  ONE  subjects  either  experienced  successful  recognition  that  they  logged  as  a  'Y' 
on  the  daily  log  sheets  or  'R'  (repeat)  or  'SEE  SECURITY'  for  unmatched  attempts  at 
retinal  scans. 

Phase  TWO  was  the  first  re-enrollment  of  Group  III.  This  provided  a  second  set 
of  patterns  in  bubble  memory  for  this  group.  At  the  onset  of  Phase  TWO  everyone  in 
the  entire  original  group  began  logging  the  PERSONAL  IDENTIFIER  that  appeared 
in  the  window  of  the  7.5  Eye-dent  on  the  daily  log  sheets.  So  Brooks  (PIN  89  and  PIN 
143)  might  be  recognized  as  either  BROOKSJ  (PIN  89)  or  BR02  (PIN  143)  depending 
if  the  daily  scan  was  matched  to  the  first  or  second  enrollment  pattern. 

Phase  THREE  was  the  first  re-enroll  of  Group  II  individuals.  At  this  point  (see 
Table  IV)  in  time,  two  thirds  of  the  original  group  had  a  second  pattern  in  the 
database  contained  in  bubble  memory.  Group  II  personnel  were  instructed  to  carefully 
log  which  pattern  was  used  in  recognition  by  writing  down  the  exact  PERSONAL 
IDENTIFIER  used  in  each  recognition  attempt. 
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TABLE  IV 

FIFTH  WEEK  LOG-IN  SHEET 

WEEK:  5 

first  trial 

second  trial 

name 

SCAN  ATTEMPT 
"the  message  or 
personal  identifier 
displayed  as  a 
result  of  the  scan 
attempt" 

SCAN  ATTEMPT 

ADCOCKJ 

adcockj 

*ad2 

AGAPIOUNE 

agap2 

**ag3 

ALDERMANB 

aldermanb 

aldermanb 

BABBR 

b3 

babbr 

*  ad2  represents  the  occurance  of  the  second  template  (1st  re-enroll) 

**  ag3  represents  the  third  template  (2nd  re-enroll) 

Phase  FOUR  was  the  second  re-enrollment  of  Group  III  subjects  with  their  third 
set  of  templates  within  the  database  of  The  Eyedentification  System  7.5  .  Quensels 
(PIN  104,  PIN  141  and  PIN  171)  could  be  recognized  against  the  patterns  QUENSELS 
(PIN  104),  QUEN2  (PIN  141),  or  Q3  (PIN  171).  Login  procedures  were  to  record 
which  of  the  three  templates  was  used  in  recognition. 
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IV.  EXPERIMENTAL  RESULTS 


Successful  recognition  during  the  study  was  the  identification  of  the  individual 
pressing  the  scan  button  from  the  eye  signature  templates  in  memory.  Identification, 
via  any  of  the  personal  identifiers,  was  considered  successful  as  long  as  the  SEE 
SECURITY  message  was  not  produced  before  the  personal  identifier.  Any  of  a 
number  of  combinations  could  be  considered  within  the  allowable  three  (3)  consecutive 
unsuccessful  attempts  which  produced  the  SEE  SECURITY  message. 

All  51  individual  records  were  examined  against  the  subjective  benchmark  success 
rate  goal  set  for  the  experiment  at  99%.  The  average  mean  for  the  entire  original 
group  of  51  was  98.29%,  which  included  all  initial  learning  trials.  By  the  end  of  three 
weeks  of  the  experiment,  all  groups  were  achieving  100%  recognition  rates.  No  false 
acceptances  (Type  II  error)  were  encountered  during  the  eight  weeks  of  the  experiment. 
All  following  discussions  will  examine  Type  I  error. 

Seventeen  (17)  participants  were  found  to  have  average  successful  recognition 
scores  below  the  99%  threshold.  Of  those  17,  two  (70%  for  PIN  99  and  80%  for  PIN 
82)  were  determined  to  be  outliers  and  were  deleted  from  the  any  computations 
involving  the  original  group.  These  two  individuals  were  omitted  for  their  consistent 
poor  performance  and  obvious  skew  from  the  entire  group. 

PIN  82  and  PIN  99  were  deleted  from  statistical  analysis  because  of  the 
questionable  nature  of  their  trials.  Consistent  poor  use  practices  of  the  retinal  scan 
apparatus  led  to  continuous  failures  on  the  part  of  PIN  99.  Repeated  instructions  and 
cautions  on  operating  procedures  did  not  produce  any  noticable  change  in  user 
methods  or  recognition  success.  A  perceived  enrollment  problem  led  to  several  weeks 
of  total  failure  on  the  part  of  the  System  7.5  to  successfully  recognize  PIN  82.  With  a 
different  eye  signature  template  of  this  same  individual  in  memory  (PIN  134),  no  other 
single  failure  was  produced  or  recorded.  Both  records  were  removed  from  the  raw  data 
prior  to  analysis. 

The  remaining  49  individuals,  as  shown  in  Figure  4.1,  used  for  experimental 
analysis,  ranged  from  92.86%  to  100%  for  total  successful  recognition  rates.  Total 
successful  recognition  rate  was  determined  by  dividing  the  number  of  successful  trials 
by  the  total  number  of  trials.  Thirty  four  individuals  from  the  experiment  had  perfect 
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or  100%  recognition  records.  Group  I  produced  thirteen  people  who  exhibited  100% 
successful  recognition  rates.  Group  II  broke  out  with  twelve  of  the  17  individuals 
exhibiting  a  perfect  recognition  rate.  Seven  of  the  thirteen  members  of  Group  III  had 
a  100%  recognition  rate.  Fifty  three  to  72  percent  of  each  group  had  a  perfect 
successful  recognition  rate. 


midpoint  of 

total  success 

count 

rate  interval 

by  percentages 

70% 

1  * 

72% 

0 

74% 

0 

76% 

0 
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•  0 
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86% 
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88% 
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0 

92% 

1  * 

94% 

0 

96% 

5  ***** 

98% 

g  ********* 

100% 

2^  ********************************** 

51  total  participants® 

the  51  success  rates  in  the  above  histogram  represents  pooled  success 

rate  for  all  conditions  over  all  attempts. 

(a),  for  illustration  purposes  onlv. 

2  questionable  data  points  (PIN  82  and  PIN  99)  were  removed  from  the 

database  so  only  49  participants  results  were  used  as  the  the  raw  data  for 

statistical  analysis 

Figure  4.1  Histogram  of  Pooled  Success  Rate. 

A  corrected  mean  for  the  49  data  points,  with  the  deleted  two  scores  of  70% 
(PIN  99)  and  80%  (PIN  82),  was  found  to  be  99.15%.  Both  the  corrected  mean 
(99.15%)  and  the  trimmed  mean  [Ref.  7:  p.  39]  for  the  original  group  (99.23%)  fell 
above  the  experimentally  determined  threshold  for  success.  The  trimmed  mean  is  a 
mathematical  device  whereby  the  top  5%  and  the  bottom  5%  of  the  sorted  raw  data  is 


23 


deleted  from  the  sample  and  the  remaining  raw  data  (90%  of  the  original  data)  is 
averaged  and  used  in  place  of  a  conventional  mean.  A  trimmed  mean  better  represents 
the  data  points  as  a  normal  distribution. 

Group  III  (the  subgroup  with  two  separate  re-enrolls)  had  a  mean  success  rate  of 
98.26%.  Group  II  (  the  subgroup  with  a  single  re-enroll)  had  a  mean  success  rate  of 
99.04%.  Group  I,  the  subgroup  with  no  additional  enrolls  beyond  the  initial  enroll  at 
the  start  of  the  experiment,  had  an  average  success  rate  of  99.23%.  These  averages 
alone  support  the  contention  that  additional  enrollments  may  not  improve  the  overall 
effectiveness  and  success  of  the  Eyedentification  System  7.5.  Whether  the  decrease  in 
success  rate  for  Groups  II  and  III  is  due  to  the  additional  number  of  patterns  or  due 
to  the  additional  retraining  and  awareness  of  the  factors  that  affect  successful 
recognition  cannot  be  distinguished  from  this  experiment.  Overall  average  success 
rates  for  all  three  groups  were  approaching  the  95  to  100  percent  rate.  Managerial 
concern  may  not  distinguish  between  the  three  approaches  so  much  as  the  overall 
success  rate  is  well  above  90%.  The  effort  expanded  to  get  a  fractionally  better  success 
rate  on  site  enters  into  the  managerial  equation  for  efficiency  and  effectiveness. 


A.  GROUP  ANALYSIS 

1 .  Group  I 

Group  I  over  the  full  eight  weeks  of  data  collection  followed  a  pattern  that 
was  relatively  predictable.  A  learning  curve  influence  can  be  seen  in  Fig.  4.2  in  that  the 
first  week  had  the  lowest  over  all  success  rate  of  97.78%.  Second  week  average  success 
rate  showed  improvement  over  the  first  week  with  98.89%.  The  third  week  showed 
perfect  total  recognition  rates  for  the  entire  group.  Unsuccessful  attempts  at 
recognition  did  not  occur  after  the  third  week  of  the  experiment  for  any  member  of 
Group  I,  II  or  III  with  the  exception  of  PIN  99,  who  was  deleted  from  the  statistical 
database.  Weeks  four  four  through  eight  averaged  100%  over  the  nineteen  (19) 
participants  studied. 

2.  Group  II 

Group  II  results  as  seen  in  Fig.  4.3  showed  an  interesting  and  distinctive 
pattern  when  viewed  over  the  full  eight  weeks.  The  first  two  weeks  had  the  same 
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PLOT  OF  GROUP  I 

o 


Figure  4.2  Plot  of  Group  I. 

overall  average  for  the  17  participants.  A  drop  in  successful  recognition  rates  occured 
during  the  third  week  for  an  average  of  95.88%.  Weeks  four  through  eight  showed  a 
consistent  success  rate  of  100%.  Interesting  to  note  is  that  the  re-enrollmcnt  for 
Group  II  took  place  during  the  fifth  week.  No  recognition  problems  were  seen  after 
the  rc-cnrollment  BUT  100%  successful  recognitions  were  occuring  the  week  before  the 
re-enrolling.  By  examining  the  averages  for  each  week  all  the  failures  occured  in  the 
first  three  weeks  with  the  least  successful  week  being  week  three. 

3.  Group  III 

Group  III  was  the  group  with  two  separate  re-enrolls;  the  first  took  place 
during  week  four,  and  the  second  re-enroll  took  place  during  week  six.  As  seen  in  Fig. 
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PLOT  OF  GROUP  II 
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WEEK 


Figure  4.3  Plot  of  Group  II. 

4.4,  week  three  showed  the  least  successful  recognition  rate  for  the  group  as  a  whole 
with  94.62%.  Week  one  had  an  overall  rate  of  93.85%,  the  lowest  single  week  for  any 
of  the  groups.  Week  two  showed  a  strong  improvement  and  some  indication  of 
learning  curve  effects  with  a  97.69%  recognition  rate.  A  noticable  drop  to  94.62% 
occured  during  the  third  week  in  Group  III.  The  week  (four)  of  the  first  re-enrollment 
produced  100%  successful  recognitions  and  this  100%  was  sustained  for  the  remaining 
weeks  of  the  experiment.  The  third  eye  template  in  memory  had  little  effect  on  the 
overall  success  rate  for  the  group. 
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PLOT  OF  GROUP  III 
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Figure  4.4  Plot  of  Group  III. 

4.  Combined  Groups 

By  transposing  the  three  groups,  as  seen  in  Fig.  4.5,  over  the  eight  weeks  of 
the  experiment  a  pattern  begins  to  emerge.  Overall  the  first  week  of  the  study  had  the 
lowest  recognition  rates  for  the  whole  experiment,  as  would  be  expected.  Introduction 
of  a  new  technology  or  a  new  security  measure  will  produce  some  trial  and  error  on  the 
part  of  the  involved  participants.  Any  drop  in  current  levels  of  access  are  directly 
attributable  to  the  learning  curve  an  individual  (and  a  new  site)  has  to  go  through  in 
mastering  new  concepts  and  protocols.  A  critical  week  for  the  experiment  occurcd  on 
week  three.  Groups  III  and  II  exhibited  a  reversed  trend  and  showed  drops  in  overall 
success  rates  during  week  three.  No  experimental  parameters  were  changed  from  week 
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PLOT  OF  ALL  GROUPS 
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Figure  4.5  Group  I,  II  and  III  Plot. 

two  to  week  three.  The  first  re-enrollment  for  Group  III  occured  during  week  FOUR. 
A  lessening  diligence  or  attention  to  procedures  could  be  one  explanation  for  the  drop 
in  success  rates  for  week  three  but  Group  I  was  already  experiencing  100%  recognition 
by  week  three,  which  tends  to  refute  any  conclusion  based  on  a  lack  of  procedures  on 
the  part  of  the  individual.  Learning  curve  trends  do  not  support  a  DROP  in 
recognition  during  week  three  on  the  part  of  any  group.  The  theory  behind  the 
learning  curve  is  the  tendency  to  keep  improving  performance  as  learning  takes  place 
and  is  acted  on  by  the  individual.  The  third  week  phenomena  cannot  be  accounted  for 
based  on  the  experimental  design. 
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B.  ANALYSIS  OF  VARIANCE 
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Figure  4.6  One-way  Analysis  of  Variance. 

Raw  data  for  Phase  ONE  for  the  three  groups  was  converted  via  arcsine 
transformation  to  a  form  that  could  be  statistically  manipulated  by  analysis  of 
variance.  Phase  ONE  transformations  were  used  because  Phases  TWO  through  FOUR 
showed  no  statistical  differences  between  the  three  groups  with  overall  success  rates  of 
100%. 
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TABLE  V 

COMPARISON  OF  WEEKLY  RATES 


Phases  of  the  Experiment 


Phase  ONE 
weeks  1-3 

Phase  TWO 
week  4 

Phase  THREE 
week  5 

Phase  FOUR 
weeks  6-8 

GROUP  I 

95.39% 

(one 

template) 

100% 

(two 

templates) 

100% 

(two 

templates) 

100% 

(three 

templates) 

GROUP  II 

97.45% 

(one 

template) 

100% 

(one 

templates) 

100% 

(two 

templates) 

100% 

(two 

templates) 

GROUP  III 

98.89% 

(one 

template) 

100% 

(one 

template) 

100% 

(one 

template) 

100% 

(one 

template) 

The  three  groups  (I,  II  and  III)  were  treated  as  three  separate  samples  and 
compared  via  analysis  of  variance  to  determine  if  the  individual  means  were  different 
from  group  to  group  or  the  difference  was  due  to  random  differences.  The  null 
hypothesis  was  that  the  transformed  means  between  the  groups  were  the  same.  The  F 
test  showed  no  difference  in  F(2,46)  between  the  group  means  and  statistically  the 
means  were  comparable  as  shown  in  Figure  4.6.  A  .05  level  of  significance  yielded  an 
F  value  of  3.23  [Ref.  8:  p.  614]  and  the  derived  F  for  the  experiment  was  0.77.  In  order 
to  reject  the  null  hypothesis  with  a  .05  level  of  significance,  the  experimental  F  value 
would  have  to  be  larger  than  the  3.23  value  taken  from  F  tables.  Therefore,  the 
hypothesis  was  not  rejected.  [Ref.  9:  p.  19]  Not  enough  statistical  data  exists  from  the 
experiment  to  point  conclusively  to  the  causes  of  the  third  week  slump  in  Groups  II 
and  III.  All  that  can  be  concluded  from  the  study  is  that  additional  patterns  did  not 
significantly  contribute  to  an  overall  increase  in  recognition  success  rates,  [Ref.  7:  p. 
197]  as  seen  in  Table  V. 
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C.  TYPE  I  ERROR 


The  learning  curve  had  an  obvious  effect  on  the  individual  attempting  successful 
use  and  recognition  via  the  EyeDentification  System  7.5.  All  recorded  cases  of 
non-recognition  (which  resulted  in  the  'see  security'  message)  occured  in  the  first  three 
weeks  of  the  experiment,  with  the  exception  of  MAUCKL  (PIN  99).  Sixty  seven  (67) 
cases  of  failure  to  be  recognized,  accounting  for  1.67%  of  Type  I  error  were  logged 
during  the  3972  experimental  trials  of  this  study.  This  1.69%  corresponds  to  the 
advertized  one  percent  Type  I  error  rate  [Ref.  4:  p.  1-6]  published  by  the  EyeDentify 
Company. 

The  1.69%  Type  I  error  from  the  study  compares  favorably  with  the  1.04%  Type 
I  error  found  by  Masiero  in  1986.  [Ref  6:  p.  34]  Although  a  moving  platform  study 
might  not  be  directly  comparable  to  the  multiple  template  hypothesis,  the  hardware  for 
the  EyeDentification  System  7.5  performed  with  strikingly  similar  results.  Based  on  the 
literature  put  out  by  the  manufacturer  and  the  Masiero  study,  the  experimental  Type  I 
error  results  fell  well  within  an  acceptable  range. 

Two  individuals  were  found  to  have  53%  (36  of  the  67)  of  the  failures  or 
unsuccessful  trials.  Both  of  these  individuals  were  re-enrolled  after  the  second  week  to 
try  and  eliminate  any  potentially  'bad'  pattern  in  memory.  One  participant  LOCH 
(PIN  82  and  PIN  134)  never  had  another  unsuccessful  attempt  during  the  course  of  the 
experiment.  This  specific  case  was  assumed  to  be  enrollment  error,  system  error  or 
system  operator  error.  The  other  case  PIN  99  had  failures  throughout  the  experiment. 
Casual  observation  of  this  individual  on  several  occasions  noted  poor  technique  and 
general  non-adherence  to  system  procedures  as  outlined  at  the  start  of  the  experiment. 
A  total  of  seventeen  (17)  individuals  experienced  some  sort  of  non-recognition  or 
failure  out  of  the  51  participants.  With  the  deletion  of  PIN  82/134  and  PIN  99  from 
the  statistical  database,  as  previously  mentioned,  the  residual  31  unsuccessful  trials 
were  spread  over  15  different  participants. 

No  unsuccessful  recognition  was  due  to  the  fact  that  any  individual  was  entered 
incorrectly  into  memory  at  the  start  of  the  experiment  and  at  no  time  during  the  course 
of  the  experiment  were  any  of  the  patterns  inoperable  or  deleted  from  the  Ibank 
(enrollee  databank).  Type  I  errors  represent  a  failure  of  legitimate,  authorized  users  to 
gain  access  via  the  retinal  scan. 
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Unsuccessful  attempts  or  Type  I  errors  are  more  an  issue  of  control  for  the 
manager,  as  compared  with  preventing  Type  II  error.  A  prime  managerial  concern  is 
the  elimination  of  Type  II  error  while  limiting  Type  I  error.  False  acceptance  (Type  II 
error)  is  the  recognition  of  an  unauthorized  user  as  an  authorized  user,  even  if  the 
individual  is'  authorized  under  his  or  her  own  identity  yet  mistakenly  identified  as 
another  PIN  or  authorized  user.  Imposters,  one  source  of  Type  II  error,  are 
individuals  not  in  the  databank  who  gain  entry  by  being  falsely  matched  against 
anothers  pattern  already  in  memory.  As  stated  before  ,  this  Type  II  error  is  the 
quintessential  security  managers  nightmare;  access  to  the  unauthorized. 

Of  the  individuals  iri  either  Group  II  or  III,  all  unsuccessful  attempts  were  logged 
before  any  second  or  subsequent  re-enrollment  took  place.  From  existing  data,  the 
existance  of  additional  patterns  in  memory  never  in  any  case  contributed  to  an 
unsuccessful  trial. 

For  most  individuals  in  the  experiment,  the  existence  of  a  third  pattern  in 
memory  did  not  significantly  affect  the  overall  chance  of  successful  recognition.  As 
stated  above  all,  failures  occured  before  the  existence  of  the  second  pattern.  Whether 
or  not  the  first  pattern  effectively  reduces  failures,  or  the  existance  of  that  single 
pattern  eliminates  additional  error  factors  that  may  be  introduced  through  the  multiple 
patterns  remains  to  be  discussed  below. 


D.  MULTIPLE  PATTERN  USE  RATES 

By  examining  the  use  of  the  different  templates  in  memory  as  logged  for  Groups 
II  and  III,  an  indicator  of  the  contribution  of  the  individual  template  toward  each 
recognition  attempt  can  be  determined.  Group  II  had  a  second  template  placed  into 
memory  at  week  five.  For  weeks  five  through  eight,  or  over  40  trials  per  individual,  an 
equal  chance  existed  for  either  of  the  two  patterns  to  be  used  in  the  recognition 
process.  For  the  seventeen  participants  of  Group  II,  the  first  enroll  template  was  used 
on  an  overall  average  of  63.65%  of  the  successful  trials  during  Phase  THREE  and 
FOUR.  One  individual  (PIN  102)  used  the  first  pattern  to  the  exclusion  of  the  second 
template  (PIN  130).  This  was  not  common  to  the  whole  group,  but  all  individuals  in 
Group  II  did  exhibit  a  tendency  to  recognize  on  one  pattern  over  another  consistently. 
The  second  pattern  was  used  in  36.3%  of  the  recognitions  logged  during  the  last  two 
phases  of  the  experiment  for  Group  II. 
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Group  III  also  had  an  equal  chance  of  using  either  of  the  first  two  templates 
during  Phase  TWO  and  THREE  of  the  experiment.  The  original  enrollment  template 
was  used  in  43.46%  of  the  trials  during  weeks  four  and  five.  There  was  no  strong 
preference  between  the  two  separate  templates.  Template  two  (or  the  first 
re-enrollment)  was  used  in  56.69%  of  the  recognition  trials. 

During  Phase  FOUR  with  an  equal  chance  of  using  any  of  the  three  templates  in 
memory,  a  stronger  tendency  to  use  a  single  pattern  emerged.  With  an  expected  use  of 
33.3%  (for  an  equal  chance)  on  any  of  the  three  templates,  the  third  template  (second 
re-enroll)  was  the  most  often  used  template  for  recognition  during  Phase  FOUR.  The 
three  weeks  of  Phase  FOUR  presented  the  possibility  of  thirty  trials  per  individual. 
Only  one  individual  (PIN  85,  PIN  146,  and  PIN  162)  did  not  use  one  of  the  three 
patterns  during  Phase  FOUR.  PIN  146  was  not  used  once  in  the  thirty  trials  of  Phase 
FOUR  as  the  template  for  recognizing  that  individual.  This  was  a  viable  template 
because  it  was  used  55%  of  the  time  in  Phases  TWO  and  THREE  to  recognize  this 
same  individual. 

The  first  template  for  the  thirteen  Group  III  participants  was  used  in  27.32%  of 
the  thirty  trials.  The  second  eye  template  (first  re-enroll)  was  used  in  25.93%  of  the 
recognition  trials  for  this  same  time  period.  The  second  re-enroll  (third  separate 
template  on  the  individual)  was  used  in  42.06%  of  all  recognition  attempts  made  by 
Group  III  during  Phase  FOUR.  Individual  logged  attempts  show  a  preference  for  a 
single  template  over  the  other  two  patterns  in  all  trials  by  Group  III,  with  the  noted 
exception  of  PIN  146. 

By  comparing  the  use  of  multiple  templates  within  Groups  II  and  III  no  strong 
statistical  trend  stands  out.  A  preference  was  shown  by  both  groups  when  using 
multiple  patterns  but  not  a  preference  based  on  sequence  of  enroll  or  use  of  template 
in  the  previous  recognition.  Template  preference  probably  is  an  indication  of  closer 
head  alignment  corresponding  to  the  template  used  in  recognition.  Toward  the  end  of 
the  experiment  several  participants  noted  the  ability  to  predict  the  template  used  based 
on  the  cant  of  the  head  on  the  fresh  scan.  A  single  template  used  for  recognition 
purposes,  may  be  more  indicative  of  the  enrolling  expertise  of  the  system  operator, 
slight  angle  variation  of  the  head  or  searching  mechanism  of  the  system  7.5  than  a 
definitive  trend  toward  successful  recognition. 

The  mechanism  by  which  the  7.5  operates  in  the  recognition  mode  lends  itself  to 
the  maintenance  of  multiple  patterns.  All  patterns  in  the  databank  contained  in  bubble 


33 


memory  are  used  for  matching  purposes.  The  five  (5)  closest  patterns  are  compared 
directly  with  the  freshly  scanned  retina  for  a  match.  With  several  patterns  in  memory 
for  one  individual,  the  opportunity  to  compare  against  several  patterns  should 
effectively  eliminate  all  Type  1  and  Type  II  error. 

Patterns  in  the  memory  of  The  EyeDentification  System  7.5  retinal  scanner  are 
not  subjected  to  'aging'  in  the  sense  of  degradation  with  use.  Multiple  patterns  can  be 
easily  collected  on  a  periodic  basis  as  long  as  memory  capacity  is  not  broached. 
Current  allocations  in  memory  allow  for  1200  patterns  at  one  time.  With  the  prospect 
of  continued  personnel  turnover,  protocols  for  eye  signature  removal  from  retinal  scan 
databanks  or  as  part  of  a  check  out  procedure  would  comply  with  any  security  or 
managerial  requirements.  Vacancies  could  be  utilized  or  reused  as  necessary. 


E.  TIME  ZONE  OBSERVATIONS 

One  interesting  outcome  of  the  experiment  was  the  occasional  time  zone  error 
encountered  by  participants  over  the  period  of  the  study.  All  time  zone  parameters 
had  been  set  to  1  (one)  to  allow  for  24  hour  access  and  to  preclude  any  possible  errors 
over  the  whole  testing  period.  At  random  intervals,  five  (5)  individuals  encountered 
"TZ  DENY"  messages  that  indicated  attempted  entry  during  an  unauthorized  time  of 
the  day  or  shift. 

After  each  occurance  of  the  "TZ  DENY"  the  system  operator  rechecked  the 
databank  to  ensure  no  corruption  or  system  operator  error  had  affected  the  databank. 
On  each  occasion,  the  databank  was  found  to  be  whole  and  complete  (as  entered 
originally  by  the  system  operator  without  any  changes  to  the  parameters).  With  ten 
(10)  separately  logged  occurances  of  the  "TZ  DENY"  message,  occuring  during  weeks 
two,  four,  five,  and  six  of  the  experiment,  the  time  zone  error  was  considered  of  value 
to  examine  along  with  other  experimental  results.  One  conclusion  is  that  the  machine 
being  used  for  the  experiment  may  have  had  a  slight  'bug'  either  in  the  software  driving 
the  hardware  or  the  system  operator  introduced  some  transparent  error  to  the 
databank  itself.  Whether  this  same  problem  has  occured  in  other  units  of  the  same 
model,  only  the  EyeDentify  company  would  have  access  to  that  information  and  can 
investigate.  In  either  case,  the  "TZ  DENY"  occurances  accounted  for  0.25%  (10 
instances  out  of  3972)  of  the  experimental  trials. 
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F.  IMPOSTER  TESTING 


Imposter  testing,  the  success  of  an  individual,  not  in  the  authorized  databank,  to 
be  falsely  recognized  as  a  pattern  in  the  databank,  was  not  formally  studied  in  the 
experiment.  Informal  imposter  testing  went  on  during  the  nine  weeks  due  to  the 
attempts  by  friends  of  participants  that  would  "try  their  luck"  on  the  retinal  scan.  No 
successful  attempt  occured  in  these  casual  attempts.  An  excellent  area  of  follow-on 
research  would  be  to  further  study  imposter  attempts  at  access  via  the  7.5  EyeDentify. 
Sandia  National  Laboratories  conducted  a  study  using  forty  one  (41)  imposters  who 
made  6520  attempts  [Ref.  10:  p.  5]  at  verification  using  both  right  and  left  single  eye 
scans.  No  false  verifications  occured  in  over  six  thousand  attempts.  A  similiar  study 
needs  to  investigate  imposter  attempts  in  recognition  mode  and  deliberate  attempts  to 
'spoof  the  7.5  System  with  retinal  pattern  sketches,  special  glasses  and  other  attempts 
at  unauthorized  entry. 
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V.  CONCLUSIONS 


In  general,  participants  were  highly  motivated  and  willing  to  use  the  7.5 
EyeDentify  during  the  experiment.  No  real  user  resistance  was  encountered  during  any 
phase  of  the  experiment.  Several  individuals  questioned  the  levels  of  light  or  daily 
frequency  required  by  the  retinal  scan.  Without  question,  most  individuals  had  not 
been  exposed  to  the  retinal  scan  or  had  limited  knowledge  of  biometric  devices.  All 
questions  and  comments  during  Phase  ONE  were  aimed  at  gaining  a  greater 
understanding  of  the  technology  and  protocols  of  use.  The  high  level  of  acceptance 
and  use  of  the  retinal  scan  during  the  experiment  can  be  partially  attributed  to  the 
military  population.  Military,  in  general,  may  be  more  used  to  using  relatively 
unknown  technology  or  devices  than  the  population  at  large. 


A.  QUESTIONNAIRE 

Questionnaires  sent  to  current  users  of  the  EyeDentify  report  no  real  managerial 
or  user  resistance  to  the  technology.  Russ  Maxwell  of  Sandia  Labs  reported  "There 
was  some  reluctance  at  first  by  several  users  who  were  concerned  about  getting  some 
kind  of  eye  disease  from  contact  with  the  rubber  cups  on  the  early  model  7.5 
EyeDentifier  used  in  our  tests"  at  Sandia  Labs.  He  also  noted  that  "About  two  or 
three  percent  of  users,  especially  those  with  severe  eye  correction,  had  abnormally  high 
false  reject  rates"  or  Type  I  error. 

Participants  in  the  experiment  were  left  to  their  own  devices  to  decide  to  use  the 
scan  by  removing  their  glasses  that  they  wore  for  normal  daily  wear  or  to  use  contacts 
instead.  It  was  noted  that  through  the  first  weeks,  users  with  the  more  heavily 
corrected  vision  tended  to  scan  with  contacts  rather  than  attempting  a  scan  by 
removing  their  glasses.  Too  much  difficulty  in  aligning  the  targets  without  their 
corrected  sight  interfered  with  the  convenience  of  the  system  of  being  able  to  come  into 
the  lab  and  scan  on  the  way  to  a  class  or  between  classes.  This  fact  probably 
precluded  more  potential  problems  with  unsuccessful  attempts.  The  convenience  of 
access  is  probably  one  of  the  stronger  selling  points  of  the  unit  and  would  be  one  of 
the  stronger  concerns  on  the  part  of  management  for  using  the  7.5  on  site. 
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All  users  contacted  (Sandia  National  Laboratories, Pacific  Telecom,  One  Safe 
Place,  Pacific  Power  and  Light,  Rockwell  International)  reported  very  good  to  excellent 
acceptance  of  the  retinal  scan  technology  by  both  users  and  management.  Units  were 
installed  for  controlling  access  to  data  center,  warehouse  or  laboratory  spaces.  After 
installation,  reported  use  has  been  expanded  to  include  time  zone  access,  substantiating 
shift  attendance  or  back-up  security  with  other  card  access  devices.  All  respondents 
were  very  satisfied  with  the  technology,  initial  operating/management  procedures  and 
vender  support. 

All  units  installed  and  in  use  were  found  to  be  effective  without  any  additional 
protocols  for  intended  site  use.  No  managerial  protocols  external  to  EyeDentify  Inc. 
had  been  developed  to  improve  effectiveness  or  increase  efficiency  for  the 
EyeDentification  System  7.5.  All  information  from  current  users  was  solicited  on  a 
voluntary  basis. 

No  other  site  had  yet  developed  additional  protocols.  One  site  (One  Safe  Place  ) 
observed  that  "the  only  problem  is  that  it  needs  to  have  another  way  of  shut-down  or 
over  riding  access  if  we  do  not  wish  someone  to  enter."  On  site  use  of  the  retinal  scan 
for  the  privately  owned  bank  vault  company  has  determined  an  additional  safeguard 
that  the  company  feels  would  be  a  good  addition  for  their  application.  This  over  ride 
mechanism,  if  ever  developed,  could  help  prevent  the  entry  of  a  false  recognition,  or  a 
Type  II  error,  to  the  controlled  spaces  of  One  Safe  Space  by  manual  intervention. 
However,  the  additional  expense  or  requirement  of  an  attendant  or  security  personnel 
to  oversee  all  attempts  would  be  a  limiting  factor  for  operation  of  the  retinal  scan  at 
other  less  unique  sites. 

The  7.5  EyeDentify  was  tested  as  a  stand  alone  device  for  effective  access  control 
via  multiple  eye  templates.  The  technology  of  the  retinal  scan  provides  the  managerial 
attractiveness  for  a  single  system  to  accomplish  security  for  specific  spaces.  Any 
cost-benefit  analysis  would  have  to  take  into  account  a  comparison  cost  of  traditional 
security  guards'  salaries  and  benefits  if  paid  by  the  company.  The  initial  costs  for  the 
unit  (in  the  range  of  S  10K)  can  then  be  compared  toward  the  prescribed  use  in  the 
organization. 
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B.  SUCCESS  RATE 


The  7.5  EyeDentify  used  in  the  experiment  recognized  the  51  participants, 
regardless  of  subgroup,  with  a  corrected  success  rate  of  98.298%.  Success  of 
recognition  was  defined  as  correct  identification  of  the  individual  on  either  the  first, 
second,  or  third  attempt  to  gain  access.  (After  three  attempts  the  SEE  SECURITY 
message  was  illuminated)  Two  individuals  encountered  a  disproportionate  amount 
(53%)  of  the  unsuccessful  attempts  in  the  study.  Removing  these  two  individuals  and 
statistically  re-analyzing  the  data  brings  the  success  rate  up  to  99.23%  which  is 
consistent  with  the  desired  predetermined  managerial  goal  of  99%  accuracy.  In  a 
manager's  world,  99%  accuracy  with  only  a  nominal  possibility  of  Type  I  error,  is  a 
powerful  selling  point. 

Group  III  with  two  additional  eye  signature  templates  in  memory  was  found  to 
have  an  overall  success  rate  of  98.26%.  Compared  with  the  99.04%  success  rate  for 
Group  II  with  only  one  other  eye  template  in  memory,  two  templates  provide  a  better 
success  rate  than  three  templates.  But  the  highest  success  levels  were  the  result  of 
using  the  single  eye  signature  template,  Group  I,  with  an  overall  success  rate  for  the 
experiment  of  99.23%.  Multiple  patterns  in  memory  were  found  to  not  enhance  the 
overall  success  rate. 

Overall  average  success  rates  were  skewed  by  the  initial  weeks  of  the  experiment. 
Each  group  had  varying  levels  of  success,  but  all  groups  were  experiencing  100% 
recognition  by  week  four  of  the  experiment.  The  100%  success  rate  was  not  dependent 
on  the  group  or  the  number  of  eye  templates  in  memory  used  for  recognition  purposes. 

For  environments  involving  small  numbers  of  users,  effective  physical  security 
can  be  managed  using  the  7.5  as  proved  in  earlier  studies.  [Ref.  5:  p.  42]  Although 
using  a  dot  eye  alignment  apparatus  as  found  on  the  current  model  of  the 
EyeDentification  System,  as  compared  to  the  daisy  alignment  system  used  in  the  Helle 
study,  several  research  recommendations  from  that  study  were  paralleled  by  the  current 
study.  An  age  range  of  15  years  was  found  to  not  affect  the  overall  success  of 
recognition.  Additional  re-enrollment  was  shown  not  to  reduce  or  lessen  the  effect  of 
Type  I  error. 

Time  zone  testing  was  ignored  due  to  the  fact  that  the  lab  area  where  the  testing 
took  place  was  only  accessible  between  the  hours  of  0700  and  1630  Monday  through 
Friday.  The  time  zone  feature  has  a  large  potential  for  use  in  a  manufacturing  or 
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operational  setting  where  shift  access  as  well  as  recognition  needs  to  be  controlled  and 
managed.  However,  due  to  the  physical  access  constraints  of  the  lab  used,  the  time 
zone  feature  could  not  be  effectively  tested. 
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VI.  RECOMMENDATIONS 


Although  the  existence  of  additional  eye  signatures  in  memory  does  not  boost  the 

/ 

overall  success  rate  of  recognition  for  a  single  eye,  the  possibility  exists  that 
improvement  may  be  gained  for  both  eyes  or  two  individuals  being  combined  into  a 
single  double  template  for  recognition.  The  EyeDentification  System  7.5  retinal  scan 
was  found  to  be  a  highly  consistent  and  reliable  device.  All  queried  current  users  of 
this  specific  system  reported  complete  satisfaction  of  application.  With  the  exception 
of  several  minor  recognition  incidents,  the  7.5  operates  at  least  as  well  as  advertized. 
Perhaps  no  better  praise  exists  than  to  say  that  something  does  what  it  is  supposed  to 
do  and  does  it  well.  For  the  eight  weeks  of  testing  at  the  Naval  Postgraduate  School, 
the  retinal  scan  7.5  was  found  to  be  so  well  designed  and  engineered  that  the  addition 
of  external  procedures  or  protocols  could  not  significantly  improve  the  mean  success 
rates  for  individuals.  In  the  managerial  world  the  difference  between  97%  and  99%  is 
often  attributed  to  luck,  which  is  just  another  label  for  random  error. 

The  EyeDentify  System  7.5  functioned  so  well  during  the  rigors  of  the  experiment 
that  any  Navy  consideration  for  site  application  would  be  endorsed  by  the  experiment 
system  operator.  With  infrequent  use  of  incorrect  procedures,  individuals  "gaming"  the 
unit  and  evidence  of  unnecessary  force  on  the  scan  button,  the  System  7.5  continued  to 
operate  in  a  consistently  superior  manner  from  experimental  design  through  eight 
weeks  of  daily  operation. 

It  appears  that,  based  on  the  experiment,  if  an  individual  exhibits  severe 
problems  using  the  retinal  scaner  during  the  enrolling  procedure,  and  initial  weeks, 
these  sorts  of  problems  do  not  improve  over  time.  One  of  the  two  individuals  that 
exhibited  the  most  problems  never  showed  any  real  improvement  over  the  eight  weeks 
and  was  deleted  from  the  analysis  for  that  reason.  Re-enrolling  the  individual  only 
showed  improvement  in  one  of  the  two  chronic  cases.  These  two  cases  represented 
only  3.9%  of  the  study  population.  In  the  statistical  study  population  with  two 
individuals  deleted,  all  remaining  individuals  were  of  equal  importance.  A  work 
environment  probably  contains  a  more  critical  core(s)  of  individuals  that  may  or  may 
not  encounter  these  same  problems.  If  critical  workers  can  not  easily  gain  access,  then 
a  whole  new  set  of  managerial  problems  have  been  introduced  into  the  workplace. 


40 


Simplifying  the  manager's  concerns  and  creating  a  workable  access  system  were  the 
original  goals  of  the  study.  As  with  any  project  under  consideration,  a  manager  has  to 
take  a  good  hard  look  at  all  the  real  and  probable  parameters  involved  in  site 
implementation. 

After  reviewing  the  results  for  this  study,  the  following  recommendations  are 
proposed  in  Figure  6.1: 


1.  Imposter  testing  in  recognition  mode  should  be  studied  based  on  the  on  the 
observations  of  this  study  and  earlier  tests.  Additional  experimentation  in  the 
area  of  'spoofing'  the  system  with  retinal  pattern  sketches  and  the  like  needs 
to  be  conducted. 

2.  Investigate  the  possibility  of  other  time  zone  errors  to  deduce  if  the  cause 
was  from  a  faulty  system  operator  or  if  other  reports  to  the  manufacturer  have 
isolated  a  possible  software  'bug'. 

3.  In-depth  investigation  of  units  already  in  the  field  for  possible  parallel  uses 
or  additional  applications  for  the  EyeDentification  System  7.5. 

4.  Greater  awareness  of  the  potential  for  'playing  the  system'  and  protocols 
for  the  managerial  safeguard  of  the  intended  use. 

5.  Study  any  possible  long  term  use  effects  of  the  EyeDentify  System  7.5. 
(hardware  or  software  failure) 

6.  Develop  mechanisms  for  incorporating  modifications  to  existing  7.5 
software  (as  demanded  by  individual  sites)  or  adapting  updated  software  as 
developed  by  the  manufacturer. 


Figure  6.1  Observations  and  Recommendations. 
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APPENDIX  A 
GROUP  III 


Group  three—  re-enroll  TWICE  during  the  experiment  (  THREE  templates  ) 
Q  (PIN  104,  141,  171) 

B  (PIN  89,  143,  169) 

G  (PIN  96,  147,  165) 

M  (PIN  85,  146,  162) 

I  (PIN  107,  135,  170) 

B  (PIN  92,  142,  163) 

P  (PIN  123,  148,  173) 

G  (PIN  115,  138,  175) 

M  (PIN  110,  137,  172) 

A  (PIN  131,  144,  167) 

O  (PIN  111,  136,  176) 

S  (PIN  124,  139,  177 
B  (PIN  126,  153,  168) 
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APPENDIX  B 
GROUP  II 

Group  two—  re-enroll  ONCE  during  the  experiment  (  TWO  templates ) 
M  (PIN  118,  155) 

N  (PIN  116,  149) 

P  (PIN  114,  157) 

W  (PIN  109,  150)  • 

A  (PIN  101,  161) 

F  (PIN  102,  130) 

R  (PIN  105,  160) 

U  (PIN  120,  152) 

H  (PIN  88,  164) 

C  (PIN  93,  166) 

A  (PIN  84,  159) 

C  (PIN  127,  154) 

S  (PIN  122,  151) 

R  (PIN  98,  158) 

K  (PIN  128,  156) 

C  (PIN  94,  145) 

H  (PIN  95,  140) 
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APPENDIX  C 
GROUP  I 


Group  one—  NO  additional  re-enroll  (  ONE  template  ) 
E  (PIN  87) 

C  (PIN  86) 

L  (PIN  113) 

M  (PIN  103) 

W  (PIN  108) 

C  (PIN  129) 

R  (PIN  121) 

L  (PIN  112) 

P  (PIN  90) 

A  (PIN  117) 

M  (PIN  119) 

K  (PIN  97) 

H  (PIN  100) 

H  (PIN  83) 

B  (PIN  83) 

D (PIN  106) 

F  (PIN  133) 

D  (PIN  81) 

P  (PIN  132) 
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